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SUMMARY 


During the grant period the following research results were obtained: 

1) Performance Analysis of NASA Telecommand System. 

Details of our work on this problem have been previously submitted 
to NASA. Three papers have been published in the open literature [1, 2, 
3]. Copies of these papers have been sent to NASA in earlier reports. A 
complete summary of this work is included as Appendix A of this report. 
This constitutes Chapter 3 of the Ph.D. thesis of Dr. Robert H. Deng, who 
was supported as a research assistant on this grant and received his 
Ph.D. from the Illinois Institute of Technology in December, 1985. 

2) Optimum Code Rate Selection in FEC Systems. 

A random coding approach was taken to determine the optimum code 
rate to use in a forward-error-correcting (FEC) system with a fixed 
signal-energy-to-noise-power-density ratio, Eb/No, but no bandwidth 
constraint* By optimal code rate we mean the code rate which gives the 
smallest decoding error probability, or equivalently, the largest coding 
gain, for a given Eb/No. A paper is being prepared on this subject for 
submission to the IEEE Transactions on Information Theory [4]. A com- 
plete summary of this work is included as Appendix B of this report. 

This constitutes Chapter 4 of the Ph.D. thesis of Dr. Robert H. Deng. 

Our results indicate, as expected, that when maximum likelihood de- 
coding is used, the optimal code rate approaches zero (infinite bandwidth 
expansion). However, for more practical bounded distance decoders, an 
optimal code rate does exist between about a rate of 0.2 and a rate of 
0.5 over a broad range for values for Eb/NQ. Calculations of code per- 
formance for several classes of specific codes tend to support these 


conclusions . 
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3) Capacity and Cutoff Rates of Concatenated Coding Systems. 

Details of our work on this problem have been previously submitted 
to NASA. Four papers have been published in the open literature [5, 6, 

7, 8]. Copies of these papers have been sent to NASA in earlier reports. 

4) Distance Growth Rates in Convolutional Codes. 

The rate of growth of the minimum distance between unmerged code- 
words in a convolutional code is an important parameter in determining 
the bit error probability of the code when used with a finite memory de- 
coder. Also, if the code is terminated to form a block code, the per- 
formance of the block code depends on the distance between unmerged code- 
words in the convolutional code. We have obtained a lower bound on the 
minimum distance growth rate between unmerged codewords for time-invari- 
ant convolutional codes. This complements a similar result previously 
obtained for time-varying codes. A paper summarizing this result was 
previously submitted to NASA [9]. 

5) Bandwidth Efficient Coded Modulation. 

Details of our work on bandwidth efficient coded modulation have 
been previously submitted to NASA. A copy of a paper on this subject was 
sent to NASA in an earlier report [10]. We are continuing our research 
in this area under our current NASA grant NAG5-557. 
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CHAPTER III 

UNDETECTED ERROR PROBABILITY AND THROUGHPUT ANALYSIS 
OF A CONCATENATED CODING SCHEME 

3.1. Description of the Scheme 

Consider a concatenated coding scheme for error control on a binary 

symmetric channel (BSC). Two linear block codes, denoted C r and C, , are 

t b 

used. The inner code C^, called the frame code , is an (n,k) systematic 
binary block code with minimum distance d^. The frame code is designed 
to correct t or fewer errors and simultaneously detect A (A>t) or fewer 
errors, where t + A + 1 _< [2] . The outer code is an (n^, k^) binary 

block code with 

n^ = mk, (3.1) 

where m, a positive integer, is the number of frames. The outer code is 
designed for error detection only. 

The encoding of the concatenated code is achieved in two stages 
(see Figure 3.1). A message of k^ bits is first encoded into a codeword 
of n^ bits in the outer code C^. Then this codeword is interleaved to 
depth m. After interleaving, the n^-bit block is divided into m 
k-bit segments. Each k-bit segment is encoded into an n-bit word in the 
frame code C^. This n-bit word is called a frame. The two dimensional 
block format is depicted in Figure 3.2. 

Decoding consists of error correction and error detection on each 
frame and error detection on the m decoded k-bit segments. When a frame 
in a block is received, it is first decoded based on the frame code C^.. 
The n-k parity bits are then removed from the decoded frame. If there 










Frame 



Figure 3.2. Block Format of a Concatenated Code Word 


are t or fewer transmission errors in a received frame, the errors will 
be corrected, and the decoded segment is error free. If there are more 
than t errors in the received frame, the errors will be either detected 
or undetected. If the errors are detected, the decoder stops decoding 
immediately and requests a retransmission of the entire block. On the 
other hand, if the errors in a frame are undetected, the decoded segment 
will be stored in a buffer and the decoder continues to decode the next 
frame. After m frames of a block have been decoded, the m k-bit decoded 
segments are then deinterleaved. Error detection is performed on these 
deinterleaved segments based on the outer code C^. If no errors are 
detected, the m decoded segments are assumed to be error free, and are 
accepted by the receiver. If the presence of errors is detected, the m 
decoded segments are discarded and the receiver requests a retransmis- 
sion of the entire block. 

The error control scheme described above is actually a combination 
of forward-error-correction (FEC) and automatic-repeat-request (ARQ) . 

In this chapter, we analyze the performance of the proposed error con- 
trol scheme. Specifically, the system reliability and the system 
throughput are considered. The system reliability is measured in terms 
of the probability of undetected error after decoding. First, by assum- 
ing the inner channel to be a memoryless binary symmetric channel (MBSC) 
with a bit error rate (BER) e^, we look at the outer channel created by 
the combination of the interleaver, the frame code, and the inner chan- 
nel. Then we develop precise expressions for both the probability of 
undetected error and the system throughput. Following that, we investi- 
gate the system reliability attainable by using random coding arguments, 
which in turn are used as theoretical guidance in the selection of inner 



and outer codes. Finally, the system performance on burst-noise- 
channels is considered. 
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3.2. The Outer Channel Model 


ff\ 

Let P*; (e.) denote the probability of correct decoding for the 

frame code. Suppose that a bounded-distance decoding algorithm is em- 
ployed. Bounded-distance decoding corrects all received n-bit sequences 
with t or fewer errors. When an n-bit sequence with more than t errors 
is detected, no attempt is made to correct the errors. Since there are 
(?) distinct ways in which i errors may occur among n bits. 


pCf) 


CO ■ I OtAl-eJ 

1 i=0 1 1 1 


n-x 


(3.2) 


for bounded-distance decoding. 

For a code word v in the frame code C^, let w(v) denote the Hamming 
weight of v_. If a decoded frame contains an undetectable error pattern, 
this error pattern must be a nonzero codeword in Let be an un- 

detectable error pattern after decoding. The probability P£(w,e^) that 
a decoded frame contains a nonzero error pattern e^ after decoding is 
given by [14,33,34] 


t min(t-i,n-w) 

Pftw.O -l l C“)C n :")c“- 1 ^tl-c i ) n - W * 1 - 3 . (3.3) 

i=0 j=0 J 


where w = w(e rt ) , and £. is the BER of the inner channel. If e. << — , 

—0 i in 


then 


nr r W» W-t,, .n-w+t 

P f (w.e i ) = C t )e. (1-e^ 


(3.4) 


,Cf) 


Let Ce i ) denote the probability of undetected error for the frame 



It 
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code. Let {A^ J , d„ < w < n} be the weight distribution of C-. 

w t — — X 

follows from (3.3) and (5.4) that 


P^PCe.) = l A Lrj P p (w,e.) , 
ud ^ i' L , w f v i J 
w=d^ 


(f), 


(5.5) 


P C ? Ce.) 
ud ^ r 


- a^^P (d e ) 

~ A d f £ La f ,e i J 

(f) n-d^+t j 

s Ai r; ( f)e. (1-t.) r , fore. « -. 
v t x v l in 


(3.6) 


Now consider any one of the m frames, say the j-th frame. If the 


decoded frame contains undetected errors, the BER e after decoding is 

3. 


given by 


= IT l A , (f) w PJw,£.). 


a n - , w 
w=d£ 


(3.7) 


For e. « — , then 
i n 


e s - d~ A^P-Cdr»e-) 
a n f d^ f f l 


dr r ~. d d.-t n-d.+t 


(3.8) 


is a good approximation to c . Let E be defined as the event that a 

a 

frame contains undetected errors. Now let e a /g denote the BER embedded 
in a decoded frame conditioned on the occurrence of event E. It 
follows from (3.7) that 


e ._ = e /P {E} = e /P^P (e.) • 
a/E a r a ud L i 


(3.9) 


For << — , substituting (3.6) and (3.8) into (3.9) yields 



(3.10) 


n d f A d^ P f W f c i ) 
' a/E = J f )p fd e ) 

d f ‘f'-W 


_£ 

n 


Now define S to be a random variable such that when h of the m 
frames contain undetected errors, and the remaining m-h frames are de- 
coded correctly, S = h, h = 0, 1, 2, • • • , m. It follows from (3.2) and 
(3.5) that 

P r {S “ h} = C h ),[P ud )(le i )]h ’ t P c f) C e i^ m " h - (3.11) 

(f) 

Note that (3.11) is not a binomial distribution because (e^) + 

( £) 

(e^) < 1, i.e., some received sequences with more than t errors are 
detected by the frame code. 

After deinterleaving of the m decoded segments (with the n-k parity 
bits removed from each frame), the BER embedded in the n^-bit block, 
conditioned on S = h, is given by 

e Q (h) = e a ^ £ • ~ , k = 0, 1, 2, • • • , m. (3.12) 

We call the channel specified by (3.11) and (3.12) the outer channel, 
and it is depicted in Figure 3.3. Note that £q( 0) = 0. This channel 
can be viewed as a block interference (BI) channel, as described in 
[35]. h = 0, 1, 2, •*•, m, is called the h-th component channel of 

the BI channel. Each block of n b bits (n fa is the length of the outer 
code) is transmitted over one of the m+1 component channels. The random 
variable S determines which component channel is used to transmit a 
given n^-bit block. 
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3.3. The Probability of Undetected Error and the System Throughput 
of the Concatenated Code 

Let {aP 3 ^ , d, < i < n,} be the weight distribution of the outer 
i d — — b 

code, where d^ is the minimum distance of C^. Let (e) be the proba- 
bility of undetected error for the outer code C, . If the n, -bit block 

b b 

is transmitted over the h-th component channel of the outer channel, 
it follows from (3.12) that 

n b n i 

pE’C'oOO) ■ I Ap ) (e Q Ch)) 1 (l-£ 0 (h)) b 1 (3.13) 

i=d b 

Let p uc jCe^) be the average probability of undetected error of the 
concatenated code. From (3.11) and (3.13) we obtain 

n=u 

= H(^)[P^ :i Ct i )] h [Pf ) Ce i )] m ' h 

h=l 

• £ b A[ b) Ce 0 (h)) X Cl-e 0 (h)) b ), (3.14) 

i=d b 

where P^ (e^ and P^(e i ) are given by (3.2) and (3.5), respectively. 

The system throughput is defined as the ratio of the average number 
of information digits successfully accepted by the receiver per unit of 
time to the total number of digits that could be transmitted per unit 
of time [8,36]. It is determined by the retransmission strategy, which 
may be one of the three basic types: stop-and-wait, go-back-N, or 

selective-repeat. All three basic ARQ schemes achieve the same relia- 
bility; however, they have different throughputs. Suppose that the 
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selective-repeat ARQ scheme is used as the retransmission strategy. The 
specific manner in which the receiver signals to the transmitter for a 
retransmission will not be considered. It will be assumed, however, 
that this backward signal is error-free, and that repeated retransmis- 
sions of a block are possible. For an analysis of various ARQ schemes 
with a noisy feedback channel, the reader is referred to Reference 8. 

For the concatenated code, let P , (e.), P (s.), and P (c.) denote 

* ud a r 1 c i 

the probabilities of an undetected error, of a block retransmission, and 
of correct decoding, respectively. Obviously, 


P .Ce.) 

ud 1 


P Ce-) 
r i 


+ P Ce.) = 1. 
c i 


(3.15) 


In the selective-repeat ARQ scheme, the transmitter sends code 
blocks to the receiver continuously and resends only those code blocks 
that are detected in error at the receiver. The probability that a 
block will be accepted by the receiver is 


P (e . ) = P (e . ) + P Ce.). 
^ l ud i c i 


(3.16) 


For a code block to be successfully accepted by the receiver, the 
average number of retransmissions (including the original transmission) 
required is 

N = 1 *P C £ i ) + 2.P(e i )(l-PCe i )) + 3-P(e.) (1-PC^) ) 2 + ••• + 


+ A-PCe^Cl-PCe.)) 


£-1 


+ * 


P(e.) P .(e.) + P Ce-) * 
v i ud^ r c^ i J 


(3.17) 


Then the throughput of the system is [2] 

1_ _ jC jb rp , . p f .. 

N ' n * n, + P c ( ' £ i )) ’ 

□ 


n = R 


(3.18) 


3.11 


k k b 

where R = — • — is the over-all code rate of the concatenated code, 
n n^ 

Note that a transmitted block will be received correctly if and 
only if all m frames are decoded correctly. Therefore, the probability 
of accepting a correct block is given by 


P c (e i ) = = C l (i)e i i (l-e i ) n " i ] m . 

i=0 


(3.19) 


For the usual situation where P ,(e.) << P (c.), it follows from 

ud v 1 c ^ l J * 

(3.18) and (3.19) that 


n „*■ • ^ • [ l 

n % iio 1 1 1 


(3.20) 


It can easily be seen that n increases monotonically as t increases, 
but that for small e^, n is only a weakly increasing function of t. 

In order to see the relationship between t and P u ^(ep, from 
(3.14) we have 

P ,(e.) = m • P^(e.) • [P^ (e. )] ra " 1 
ud^i' ud ^ l 1 c ^ l J 


• { l A< b) (e (D^fl-e (I)) ° }, for e « K (3.21) 

i=d b 

Using (3.6), (3.10), and (3.12), P ud ( £ p can be further approximated as 

d • d -t n-d _+t r ^ . 

P ud (ei) = K • ( pc. (1-e.) • [Pp , (e i )] ' , (3.22) 


where 


K = m • • ( 'l A^^-ftl - ^-) V S 

d- . L , i v m*n m*n 

f 


is a constant which is independent of t. Let Q(t) denote the right 
hand side of (3.22). Then 
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Q(t+i) 

Q(t) 


Cd f -t) 

"Tt+TT 


>> 


£ . 
1 


n 


for £ . 

1 


1 

<< — . 
n 


(3.25) 


That is, for e^ << 1/n, when t increases by 1, p uc j^ £ i^' t ^ ie probability 
of undetected error, will increase by approximately e^. Thus P^(e^) 
is a strongly increasing function of t. For this reason, a large value 
of t is not desirable in such a system. 


3.4. A Bound on the Reliability of the Concatenated Code by _a Random 
Coding Argument 

In this section we derive a lower bound on the reliability of the 

concatenated code by using a random coding approach. Although the bound 

may not be tight enough, especially when the inner code is used for 

error detection only (i.e., t=0), it does give some insight into the 

concatenated code. 

(fl 

Let P^ •'(e.) be the probability of decoding error for the frame 
code when the frame code is used only for error correction. Again, let 
P^^p(e^) and P^^ (e^) be the probabilities of undetected error after 
error correction and of correct decoding for the frame code, respective- 
ly. Obviously, 

P u? tC i ) i P E f)fe i>- (3 - 24) 

Then p uc jC £ ^)> the probability of undetected error of the concatenated 
code, from (3.14) and (3.24), can be bounded by 
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P ud (c i ) = j/hH P ud ) ^i ) l h t P c f, ' E i ) l”' hp S’( E 0 (h » 
h = l 


m 


i h V"’' P ud ) ' e i”'’ P ud ! ' E 0 (h » 

n=l 


m 


ij,O[ p f< £ i!) hp S ) ( £ 0 <h > ) 

h=l 


(3.25) 


Let r and denote the ensembles of inner systematic codes and 
of outer systematic codes, respectively. We assume that the two en- 
sembles of codes are selected independently of each other. Then the 
average probability of undetected error over * s » ^ rom (3.25) 


P , (e. ) < 
ud v x — 



hp(b) 

ud 


(e 0 (h)). 


- J OtPf ) Cc i )] h • 00). (3.26) 

h=l 

where the first average is over and the second over r~ . Equation 
(3.26) can be rewritten as 


m 


Vh’ 

h=l 




h D Cb) 


t P E ^ P ud 


(3.27) 


because the average of the sum is equal to the sum of the averages. By 
the memoryless assumption of the inner channel, we also have, from 


(3.27), 




00 ). 


(3.28) 


The last term in (3.28) is given by [2] 
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p S ) ( £ o th »i 2 “ " = 2 


OvV -v^v 


(3.29) 


which is independent of h, where R ? = k^/n^ c0< ^ e ra ' te °f the 

outer code. Using this fact, equation (3.28) yields 


p ud<n> i 2 


-Vi-V 


m 


l 0 ip 

h=l n 1 


(3.30) 


Now we define the reliability of an (n,k) code as 


E > - lim — log 0 P ,. 
— _ n 6 2 ud 

n-*» 


(3.31) 


For the concatenated code, the reliability is 


E> - lim— logjW- 
n^-x» d 


(3.32) 


Note that n^ = km = R mn. For fixed R 1 and m, n^ going to infinity is 
equivalent to n going to infinity. Hence (3.32) can be rewritten as 

-v i -v 


E >_ - lim — log 2 2 

V“ " b 


- llm sr? 10 S2 { J, 0 [ p E £) ( £ iH hl 


n-*» 1 


h=l 


■ ^ *iT t- “-W 10 *: p E f) cy 

1 n-*» 


limi log 2 [m . J O [Pj'Up] 1 

TL-X° h = 2 


m 


roCf) ( r- 


(3.33) 
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Because ^(e^) -*■ 0 when n-* 30 if and only if R^ < C, where C = 

1 - H(e^) = 1 + e^log^c^ + (l -c j L ) (1”-^) is the inner channel 
capacity [15], the last limit in the brackets {•••} goes to zero. 
Therefore 


E i H 2 *557' - U "1T l0 % p E f)ce i ) 1 


n-*» 


= 1-R. + -i- E,, 

2 mRj f 


(3.34) 


where 


A 


1 f f 1 

lim — log Pp •*(£.) is given by [15] 

n-*co n z 1 


where 


E f - H 


X log. 


Me. (1-e. ) 
l i 


R o - R 1 


E(X, e.) 


> °i R i!V 

* \ - R 1 - R C > 


> R c 1 R 1 1 C ’ 


X = H -X (1-R 1 ) , 

E(X,e.) = H(c.) + (X-e^H’ce.) - H(X), 


R = 1 - H( — ), 

y 1 . /a 


l + /4e . (1-e . ) 

l l 


and 


R c = 1 - H( 


/ G i 


), 


e . + / 1 -e . 
i l 


R 0 = 1 - iog 2 (i + 2/E-(I^7T). 


(3.35) 


(3.36) 

(3.37) 

(3.38) 

(3.39) 

(3.40) 


1 


(3.41) 
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The reliability E, bounded in (3.34) is plotted in Figures 3.4 
- 2 -4 

and 3.5 for = 10 and = 10 , respectively. These results show 
that low inner and outer code rates are needed in order to obtain high 
system reliability. 


3.5. Examples 

Having investigated the theory of the concatenated code in the pre- 
vious sections, in this section we present some concatenated code ex- 
amples whose purpose is to give a feeling for the actual system perform- 
ance. Recall that the concatenated coding scheme described above is 
used in ARQ systems, and that the major advantage of ARQ is that it 
requires simple decoding equipment, while achieving high system relia- 
bility and throughput. Therefore, only those codes which require simple 
decoding are chosen as examples. We should point out that the results 
of sections 3. 3-3. 4 are a useful general guide to code selection. 


Example 3.1 

This concatenated code example has been proposed for a NASA tele- 
command system. The frame code is a distance-4 Hamming code with 
generator polynomial 

g(x) = (x+1) (x 6 +x+l) = x 7 + x 6 + x 2 + 1, (3.42) 

where x^ + x + 1 is a primitive polynomial of degree 6. The natural 
length of this code is 63. This code is used for single error correc- 
tion (t=l), and is also used to detect all error patterns of double 
weight and some higher odd weight error patterns. The outer code is a 
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distance-4 shortened Hamming code with generator polynomial 

( , f 1W 15 14 13 12 4 5 2 

g(x) = (x+l)(x + x + x +x +x + x + x + x + 1) 

16 12 5 ... 

= x + x + x + 1, (3.4 j) 

, 15 14 13 12 4 3 2 . . ... 

where x + x +x +x +x +x + x +x+lisa primitive poly- 
nomial of degree 15. This code is the x.25 standard for packet-switched 
data networks [37]. The natural length of this code is 2* -1 = 32,767. 
In this example, a shortened code of maximum length 3,584 bits is con- 
sidered. This code is used for error detection only. 

We assume that the number of information bytes (IB) in a frame is 
between 3 and 7, that is, the inner code can also be shortened. The 
number of frames in a block is between 4 and 64. 

To obtain a precise result for P ^(e..), a computer program was 
written to help determine the reliability of the proposed concatenated 
coding scheme. We found that if only one frame contains a weight 4 
undetected error pattern, then this error pattern can always be detected 
by the outer code. Thus (3.14) can be modified as follows: 


P ,(e.) = (T)P < '5' ) C^. 3P < - £ - ) (e. ) m_1 

ud^ r v lud v ic 

% i 

• l A. (b) (7 0 (i)) i (i-7 0 (i)) nb 1 
i=d b 


+ l (0[P^Ce i )] h [pf ( e i ) ] ra - h 

h_2 

% i 

• I A Cb) (£ (h)) 1 (l-£ (h))^ 1 }, 

1=d b 


(3.44) 


where 
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and 



L , w f 1 ' * 


w=d^.+ l 



(3.45.1) 


(3.45.2) 


Results for the probability of undetected error P u ^(e^), based on 
(3.44), and the system throughput n are plotted in Figure 3.6, where we 
have used the method in [6] to obtain 

P ud ) t E 0 (h » = .T A i b) ( £ 0 <W)ici ' £ 0 Ch)) ' lb 1 

1=d b 

The system described above can be altered by allowing the frame 
code to do error detection only (i.e., t=0). In this case, P ^(e..) and 
n are shown in Figure 3.7. 


Example 3.2 

The same frame code and outer code are employed as in Example 3.1. 
The inner channel is, however, assumed to be a AWGN channel with BPSK 
modulation and the frame code is decoded by using the Viterbi decoding 
algorithm with repeat request and infinite demodulator output quantiza- 
tion [38]. Let u, a positive real number, be the retransmission metric 
threshold of the algorithm [38]. Let P^^p , P^^ , and denote the 
probability of undetected error, the probability of detected error, and 
the BER after decoding, respectively, for the frame code. Then [38] 





m=64, IB=7 



£ i 


Figure 3.7. Performance of the Concatenated Code of 
Example 3.1 (t=0) 
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,/N rt is the channel symbol signal energy-to-noise power density 
4 0 


. From (3.46) and (3.47) we see that the probability of correct 
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decoding for the frame code is 


P 


(f) = 

c 



= 1 




y 


for 



<< 



(3.52.1) 

(3.52.2) 


The probability of undetected error of the concatenated code, P and 
the system throughput, n, can be computed by using (3.46)- (3.52 .2) in 
the formulas given in section 3.3. Both P ^ and n are shown in Figure 
3.8 for u=4 with respect to e^, where E^/Nq and are related by the 
equation 

2E \i 

e « Q( / -£■ ). (3.53) 

w 0 

The influence of the value of u on the system performance is 
obvious. For larger values of u, from (3.46), (3.47), and (3.52.2), 
the probabilities P^p and become smaller, and consequently the 

probability of undetected error and the system throughput are lower. 


Example 3.3 


The outer code is again a shortened distance-4 Hamming code with 
generator polynomial given by (3.43). The frame code is an (n,n-l) 
single-parity-check code. The frame code has a minimum distance of 2, 
and is used for error detection only. The frame code can detect all odd 
weight error patterns. The weight distribution of the frame code can be 
calculated from 


^ = ( n "^) + ( n ”S 

2i 1 2i j l 2i-l J 

= 0, for all odd j. 
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(3.54.2) 



Figure 3.8. Performance of the Concatenated Code 
of Example 3.2 with u=4 






where (£) = 0 for k < 0 and k > n, and [xj denotes the integer part of 

x. 

Because the outer code detect three or fewer errors, if only one 
frame contains a weight 3 or less undetected error pattern, then this 
error pattern can always be detected by the outer code. Hence, equation 
C3.44) is used to compute the probability of undetected error, where 
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17J 
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is the probability of undetected error when the undetected error 
pattern has weight greater than 3, and 
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Figure 3.9 shows the probability of undetected error P .(e^) and the 
system throughput n for this example. 

From Figures 3. 6-3. 9, we observe that the performance of a particu- 
lar scheme depends strongly upon the channel noise conditions. There- 
fore, we cannot say that a particular one of the above schemes is 
"best". However, we can draw a number of conclusions which will be dis- 
cussed below. 

From Figures 3.6 and 3.7 we can see the tradeoffs between the 
probability of undetected error and the system throughput obtained by 
varying the number of correctable errors t in the frame code. Smaller 
value of t always result in a lower probability of undetected error. 


Figure 3.9. Performance of the Concatenated Code of 
Example 3. 





and therefore a higher system reliability. But as the channel BER gets 
higher, the system throughput degrades rapidly for smaller t. The 
system throughput is less affected by t if the channel BER is small. 

Figure 3.8 shows the advantages of a Viterbi decoded frame code 
with repeat request over an algebraically decoded frame code. The 
Viterbi decoding algorithm makes the system much more flexible in trad- 
ing between system reliability and throughput by simply changing the 
value of u. Varying u can be viewed as a generalized method of "varying 
t" for algebric decoding of the frame code. From comparison of Figures 
3. 6-3. 9 we see that lower inner code rates provide higher system-re- 
liabilities but lower system throughputs. 

In Figure 3.10 we plot P vs. n f° r 'the above examples with 

m = 64 and IB = 7. The infinite slope of the curves is due to the fact 
that at low channel BER’s the system throughput becomes saturated. We 
conclude that, at moderately low BER's, the concatenated coding scheme 
is capable of achieving high system throughputs and extremely low unde- 
tected error probabilities. 


3.6. The Concatenated Code Performance on a Burst-Noise-Channel 

Channels with memory often occur in practice. Errors on these 
channels tend to occur in bursts, and hence they are called burst -noise - 
channels . Examples of burst-noise-channels are radio channels, where 
the error bursts are caused by signal fading due to multipath transmis- 
sion, wire and cable transmission, which is affected by impulsive 
switching noise and crosstalk, and magnetic recording, which is subject 
to tape dropouts due to surface defects and dust particles. In this 
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section we extend the performance analysis of the concatenated coding 
scheme to burst-noise-channels. The results here will be compared with 
those presented in the previous sections. 


3.6.1. The Inner Channel Model 

The generalized Gilbert type channel [11,39,40], as shown in 
Figure 3.11, is used as our inner channel model. There are two states 
in the model. Each state is a BSC. State 1 is the "quiet" state, where 
the BER is e^. State 2 is the "noisy" state, where the BER is and 

e 2 >:> e i* '^ ie trans rtion probabilities between states are P = P^t l-*2} 
and p = P^{ 2 1 } [see Figure 3.11). Q = 1-P and q = 1-p are the proba- 

bilities of remaining in states 1 and 2, respectively. To simplify the 
model's treatment, we assume that one transition time in the model 
corresponds to the transmission of one frame of length n bits, i.e., the 
noisy bursts last for a multiple of the transmission time of a frame. 

The average burst length is then [11] 

L = frames, (3.56) 

or 

I = Ln = - n bits. (3.57) 

P 

The average BER is 

7 = (pe x + Pc 2 ), (3.58) 

and the probability of being in the noisy state is 



(3.59) 
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Four parameters govern the model. They can be chosen to be L, e, p 7 , 
and the high-to-low BER ratio 


5.6.2. The Outer Channel Model 

Let C e j) > P ud^ £ j^’ e a j and e aj/E* ^ = l > 2 > denote the proba- 
bility of correct decoding for the frame code, the probability of unde- 
tected error for the frame code, the BER in a decoded frame, and the BER 
embedded in the decoded frame conditioned on the decoded frame contain- 
ing undetected errors, respectively, when the frame is transmitted in 
state j. (In the following we will always use the subscript j, j = 1, 

2, to denote that a frame is transmitted in state j.) Then P^(e.), 

C e j) > e^, and e a ^ /E are given by (3.2), (3.5), (3.7), and (3.9), 
respectively, with replaced by c ^ , j = 1, 2. 

Now define E„ , , 0 < l < h < m, to be an event such that h of the m 
£ , h — — — 

decoded frames contain undetected errors (the other m-h decoded frames 

are error free) and £ of the h containing undetected error frames are 

transmitted in state 2 of the inner channel. Let P {E. . } be the prob- 

r z , n 

ability that event E^ ^ occurs. Then after deinterleaving of the m 

segments (with the n-k parity bits removed from each decoded frame), the 

BER embedded in the n, -bit block, conditioned on the occurrence of event 

b 

E^ h , is given by 

e 0 (E £,h :) = t £ * e a2/E + e al/E 1/m * 0 <_ £ < h <_ m. (3.60) 

We call the channel specified by (3.60) and the probability distribution 

P (E ,} the outer channel (see Figure 3.12). 
t z j n 
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3.6.5. Performance of the Concatenated Code on a Burst-Noise-Channel 


If the n, -bit block is transmitted over the component 
b 




£,h 


of the outer channel, the probability of undetected error of the outer 
code is 

n b 

p ud ) ^o (E t.h» ■ Vi b) t' 
l=d. 


.)) 


V 1 


(3.61) 


With the aid of the above outer channel model, the average probability 
of undetected error of the concatenated code is 


ud 


m 

- I 


h 

l 


h=0 £=0 


V^.h^SS^.h”- 


(3.62) 


For large m, the computation of (3.62) is very complex and time 
consuming. To reduce the computational work to a manageable amount, 
we seek an approximation to (3.62). 

Define 


e — max [e , , e J • 

max 1 al/E’ a2/E J 

It follows from (3.60) that 

e n (E 0 < h £ /m.'.= e n (h) 

0 £,h — max 0 


(3.63) 


(3.64) 


and equality holds when and £2 are equal, i.e., the inner channel is 
a memoryless BSC. Assuming that P^ ( 2 ) is an increasing function of z, 
0 < z < 1/2, we obtain from (3.62) and (3.64) 


P ud - J n J P r {E A,h }P S )c£ 0 Ch)) 


h=0 l =0 
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£ = 0 


* * J„ P S ) ^o Ch » 6Ch >' C3 - 65 > 
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where 

h 

8(h) = l P(E }, 0 < h < m, 

£ = 0 r 5 


(5.66) 


is the probability that h of the m decoded frames contain undetected 
errors (and the remaining m-h decoded frames are error free). 6(h) can 
be readily computed by a recursive method. To find 6(h), we model the 
decoded frame status as a Markov chain (see Figure 3.15). In state j, 

3=1,2, the decoded frame contains an undetected error with probabil- 

( f ) ( f ) 

lty PV^ (£j) and is error free with probability P^ (e_.) . 

Define G(h,m) = P^Oi of the m decoded frames contain undetected 
errors / the inner channel starts in state 1} and B(h,m) = P r (h of the m 
decoded frames contain undetected errors / the inner channel starts in 
state 2}. By applying a similar argument as in [40], we have 

6(h) = C(h,m) + ^ B(h,m), 0<h<i. (3.67.1) 

G(h,m) and B(h,m) can be found recursively from 

G (h , m) = G(h,m-1)Q P^ f) (e^ + B(h,m-l)P P^ f) QeJ 

+ G(h-1 ,m-l)Q + B(h-1 ,m-l)P P^Ce^, (3.67.2) 

B(h,m) = B(h,m-l)q P £ f) (e^ + G(h,m-l)p P^ (e^ 

+ B(h-l,m-l)q P^ } (e 2 ) + G(h-l,m-l)p P^ } (e 2 ) , (3.67.3) 

GC0.1) = P^Cej) , B(0,1) = P^ f) (c 2 ), 

G(l,l) = P^Cep » B(1 , 1) = P^f(£ 2 ) (3.67.4) 


we must also assign the values GCh,m) = B(h,m) = 0 when h < 0 or h > m. 
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Note that if = e a' : ’/E , t ^ ie u PP er bound of (3.65) is very close 

to (3.62). Fortunately, this is usually the case for 0 < < 1/2, 

especially for small e and e_, for then s . a d-/n. 

x z ci 1 / 1* r 

Although in general the computation of (3.62) is very involved, in 
the following two important cases it can be handled quite easily. 


1) = 0. That is errors are not allowed to occur in state l.Then 

(3.62) reduces to 

m rv,i 

P ud * J n P r {E h,h }P id C £ 0 CE h,h^’ C3 * 68 * 1) 

h=0 

where 

■ h e a2/E ' »• 0<h<m, (3.68.2) 

P r (E h,h :) * p?p G 0'- m l * pTJ 0 < h _< m, (3.68.3) 

and both G(h,m) and B(h,m) can be found from (3.67.2)-(3.67.4) by 
letting P^Cep = 1 and P^ (e^) = 0. 


2) P = 1-p, i.e., the inner channel of Figure 3.11 becomes a BI 

channel, as shown in Figure 3.14. P^ and are the probabilities 
of being in states 1 and 2, respectively. The probability P^tE^ 
is given by 


m 


P r <E «,h } = l 'OOt P id ) ' £ 2 ’ <1 1 l p ' f) ( e2 ) 5 - ,l ]P 2 S 
s ^ / 




s [ f, ud )tE 2l t ( P id ) t £ Pl h ' , 'l P 2 S Pj"' 5 , 


s = £ 


0 < £ < h < m. 


(3.69) 
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can be computed by using (5.69) in (5.62). 

To evaluate the system throughput, again assume that selective- 
repeat ARQ is used. Let P^, P r > and ? c denote the probabilities of an 
undetected error, of a block retransmission, and of correct decoding, 
respectively. Of course. 


P.+P + P =1. 
ud r c 


(3.70) 


In order to simplify the problem, we assume that retransmissions do 
not depend on the previous inner channel states. This is a reasonable 
assumption if the channel round-trip delay is large. Then the through- 
put of the system is [2] 


and 


n = - — (1-P ) =--(P ,+P ) 
n r n ud cr 


_ js. p 

~ n "b c # 


p =r - 2 — G (m) + rr 

c P+p P+p 

G(m) = G(m-1)Q P^ f) 

B(m) = B (m-l)q P^ f) 

G(l) = P^ f) (e 1 D , 


B(m), 

(ep + B(m-l)P P^ f) 
(e 2 ) + G(m-l)p p£ £) 
BCD = ( e 2 ) • 


Ce 2 ), 


(3.71) 

(3.72) 

(6.73.1) 

(6.73.2) 
(6.73.5) 
(6.73.4) 


3.6.4. Examples on a Burst-Noise-Channel 
Example 3.4 


The same frame and outer codes are used as in Example 3.1. The 
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probability of undetected error, P and the system throughput, n, are 
plotted in Figures 5.15(a) and 5.15(b) for t=l, and in Figures 5.16(a) 
and 3.16(b) for t=0, respectively. 


Example 5.5 

The same coding scheme is used as in Example 3.3. P ^ and n are 
shown in Figures 3.17(a) and 3.17(b). 

The performance of the concatenated coding scheme on burst-noise- 
channels heavily depends on the channel's parameters, especially on the 
high-to-low BER ratio, As shown in Figures 3.15 (a)-3. 17(b) , for 

a given average BER e", with other parameters fixed, as the rat ^° 

becomes large, the system performance becomes poor. Our results indi- 
cate that on a burst-noise-channel for the same average BER, the system 
reliability degrades greatly, while the system throughput remains almost 
the same, compared with the same coding schemes on a memoryless BSC. 

For moderate values of average BER, high system reliability and through- 
put are still achievable using the concatenated coding system on burst- 


noise-channels . 


Figure 3.1SCa). Performance of the Concatenated Code of 
Example 3.4 with p. = 0.1, L = 5, e./e, = 10, t = 1 
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CHAPTER IV- 
ON THE OPTIMAL CODE RATE IN AN FEC SYSTEM 

4.1. Problem Statement 

A problem which frequently arises in forward-error-correction (FEC) 
coding is how to select a code from among various existing codes to ob- 
tain the best system performance. More specifically, what is the opti- 
mal code rate and code length that should be chosen? 

Consider a discrete memoryless channel (DMC) with Q possible 
channel inputs x = {a^, a^, •••, a^}, which are specified by a distribu- 
tion vector £ = q{(a^), q(a 2 ), •••, qCa^)}, subject to the constraints 

q(x) £ 0 , for every xeX, 

and 

IqCx) =1. (4.1) 

X 

The channel transition probabilities are denoted by p(y|x) for every 
output yeY and input xeX. For such a channel, the Gallager function 
is defined by [22] 

1 

EqCp > £) = -log 2 l [£q(x)p(y|x) 1+p ] 1+p , 0 < p < 1, (4,2) 

Y x 

and plays a central role in the channel coding theorem. The channel 
transition probabilities p(y|x) in (4.2) depend on a parameter E^/Nq, a 
nonnegative real number, called the channel symbol signal energy-to- 
noise power density ratio. When coding is used, if R is the code rate, 
as defined in Chapter II, then 



where is the information bit signal energv-to-noise power density 

ratio. For a given communication system, the average transmitter power 
is fixed, and so is E^/N^. Therefore, changing the code rate R varies 
the channel parameters. Most previous work on error control coding has 
not considered the influence of code rate on the transmission channel. 

As a result, a communication engineer who must design a practical 
coded communication system may find little theoretical guidance in 
selecting the best code rate for the system. In this chapter we attempt 
to partially remedy this situation by finding the optimal code rate in 
an FEC system. By otpimal code rate, we mean the code rate which gives 
the smallest decoding error probability, or equivalently, the largest 
coding gain for a given E^/N^ and code length (for block __ codes) or con- 
straint length (for convolutional codes) . We begin by considering ran- 
dom coding error probability bounds [21,22] as a measure of system 
performance, and then proceed to consider random bounds on minimum dis- 
tance. Both kinds of bounds will be used to study the optimal code rate 
problem. 


4.2. The Optimal Block Code Rate 

4.2.1. The Optimal Code Rate in Terms of the Error Probability Bound 

The channel coding theorem says that the average decoding error 
probability over the ensemble of all block codes of length n and rate R 
for the Q-ary input DMC described in section 4.1 is bounded by [21,22] 
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where 

E CR) = max max [E n (p, g) - pR], (4.5) 

1 °l°l l 

and E q (p, g) is given by (4.1). It follows from (4.4) that at least one 
code in the ensemble must have Pg no greater than this ensemble bound. 

Note that E q Cp, g) is also a function of R. If we use the code 
length n as a measure of decoding complexity, from (4.4) and (4.5) we 
see that the code rate R should be chosen such that E^(R) is as large 
as possible. Formally, the optimization problem is 

maximize: E- (R) 

b 

subject to: ^(R) > 0. (4.6) 

The code rate R which satisfies (4.6) is defined as the optimal code 

rate and is denoted by R^l. 

opt 

Another important quantity in describing the performance of coded 
communication systems is the computational cutoff rate Rq. It is de- 
fined as the largest numbeT for which there is a bound with a linear ex- 
ponent, i.e., a bound of the form [21,22] 

-n(R -R) 

P E < 2 0 , (4.7) 

on the average decoding error probability of all codes of length n and 
rate R on the Q-ary DMC described in section 4.1, where 

R q = max E Q (1, g) 

a 

= max{-log 2 £[£q(x)/p(y|x) ] 2 }. (4.8) 

a y x 


If we define 
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Eb 0 M - R 0 - R. 
the optimization problem is 

maximize: E^q(R) 

subject to: E^q(R) > 0, 

and the code rate R which satisfies (4.10) is denoted by R 


(4.9) 


(4.10) 


(b0) 

opt 


It can be seen from (4.1), (4.5), and (4.8) that as long as p(y|x) 
is continuous in R, both E^(R) and E^q(R) are continuous functions of R 
over a closed region. Therefore, there exists at least one value of R 
which satisfies (4.6), and at least one value of R which satisfies 
(4.10). More general statements about (4.6) and (4.10) cannot be made 
because the channel transition probabilities p(y|x) depend on R, E^/N^, 
the modulation/demodulation scheme used, the channel noise characteris- 
tics, the channel output quantization method, etc. This fact is made 
more clear by the following examples. 


Example 4.1 

Suppose that BPSK or BFSK modulation is used over an AWGN channel. 

If the demodulator makes hard quantization, a BSC results with bit error 

rate e = efR^/NQ), where e is a decreasing convex U function of R, as 

shown in Appendix A. From (4.1) and (4.5) we obtain 

JL _L 

1 + Pq 

^(R) = P 0 - (1+P 0 )log 2 [ £ + (1-e) U ] - P 0 R 

= E q (p 0 ) - p Q R, (4.11) 


where 
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1 


1 


1+p 0 1+O 0 

E 0 (P 0 ) = P Q - (l+P Q )log 2 [e + (1-e) ], 


and p Q is the value of pe[0, 1] which maximizes E^( R ) 
show that 


and 


d W 

dz 


de 2 


<0 , 0 < e < 4 , 


> 0 , 0 <£<•=■ • 


It is easy to 


(4.12.1) 


(4.12.2) 


We also have 

dEh(R) 


and 


dR 


dE 0<V de 

de dR p 0 ’ 


d V R > d E 0 Cp 0> , de , 2 d W d 2 _s 

.2 


dR 


de 


C u. c, v 

Hr’ * 


de 


dR 


(4.13.1) 


(4.13.2) 


Since 

that 


= (R*E^/Nq) is a decreasing convex U function of R, it follows 


de n 
dR < ° 


iz 

dR 2 


> 0 


, R > 0 , 


R < 0 . 


(4.14.1) 

(4.14.2) 


Now let C(R) denote the channel capacity, which is a function of 
the code rate R. From (4.12.1)- (4.14.2) we can draw the following con- 
clusions : 


1) 


V R ’ . „ . d W , de 2 


dR 


2 - 


< 0, if 


de 


f — ) < 

^ dR J - 


d V p) d^e 

.2 


de 


dR 


for 0 < R < C(R) , 
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and the optimal code rate is the unique solution of the follow- 

ing equation 


2 ) 


d W 


de 

dR 


= P 


O’ 


d \ m . „ ir d2 w 

9 > U , 1 X 9 


dR 


de 


( — ) 
^ dR J 


d 2 e 


de 


dR 


(4.15) 


for 0 < R < C(R) , 

and 

R tb ) = R* , 
opt * 


(4.16) 


where R maximizes C(R ); 

3) If E^(R) is not a convex function for 0 _< R £ C(R), R^j^ must be 
found numerically. 


By replacing p Q with 1 and C(R) with R Q , respectively, similar conclu- 
sions apply to the function E^q (R) • 

This example is used to indicate that even for the particularly 
simple, often studied, BSC, finding the optimal code rate R^j^ is rather 
complex. The computation of R^| for explicit channels is generally 
very involved, because the function E^CR) (or E^g (R)) depends on R, 
E^/Ng, the modulation/demodulation scheme used, the channel noise char- 
acteristics, the channel output quantization method, etc. Fortunately, 
direct numerical computation can be carried out easily by computer. The 

optimal code rate R^ for BPSK modulation over an AWGN channel with 
r opt 

demodulator output hard quantization is given in Table 4.1 and depicted 
in Figure 4.1 as a function of E^/Ng. 
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Table 4.1. R^ b ^ for BPSK Modulation over an 
opt 

AW GN Channel with Output Hard Quantization 


V N o 

R< b J 

opt 

C 

E K CR Cb i 

b opt 

1.1 (T) 

0 

0 

0 

1.2 

0.06 

0.064 

0.0001 

1.5 

0.17 

0.209 

0.0024 

2.0 

0.27 

0.392 

0.0121 

2.5 

0.32 

0.522 

0.0388 

3.0 

0.34 

0.610 

0.0691 

4.0 

0.37 

0.746 

0.1429 

5.0 

0.38 

0.828 

0.2238 

6.0 

0.38 

0.879 

0.2938 

7.0 

0.37 

0.910 

0.5520 

8.0 

0.36 

0.931 

0.4010 

9.0 

0.34 

0.942 

0.4420 

10.0 

0.33 

0.954 

0.4780 




Optimal Code Rate 
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Figure 4.1. Optimal Block Code Rate for BPSK Modulation over an 
A.WGN Channel with Output Hard Quantization 
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Example 4.2 


For M-ary PSK modulation over an AWGN channel with no demodulator 
output quantization, the computational cutoff rate is given by [23] 

i M E, . . 

R 0 = * log 2 M I exp[-R ~ sin ~] , (4.17] 

4 k=l 1 0 M 

where M is a power of 2. It can be shown that [see Appendix B] 

, for E^Ng > 0 , R >_ 0, (4.18) 

, for ^/Nq _> 0 , R_>0. (4.19) 

Equality in both (4.18) and (4.19) holds only when E^/N^ = 0. It 
follows from (4.19) that 

<0 , for E./N. >0 , R > 0. (4.20) 

dR“ ° 0 



dR 0 n 
dR - 0 


and 


dR^ 


Therefore, E^q(R) = Rq - R is a strict convex n function of R > 0 for 
E^/Nq > 0. The equation for a stationary point of E^q(R) with respect 
to R is 

dR„ 


dR 


- 1 = 0 . 


(4.21) 


Since d 2 R Q /dR 2 <_ 0, 


ject to E bQ (R) _> 0, 
,(b0) 

opt * . : • 

If M=2 and 4, 


rate R 


any solution of (4.21) in the range 0 <: R ^ 1 , sub- 
maximizes E^q(R), and hence gives the optimal code 

i.e., BPSK and QPSK modulation, the optimal code rate 


can be found explicitly by solving (4.21) and is given by 


4.10 


opt 


1 *n[ — / n2 }> for BPSK, E b /N Q > 2*n2, 


E b /N 0 1 V N 0 ' £n2 


(4.22) 

E b /N Q E^Nq - £n2 ^ f ° r QPSK> E b /N 0 - 2£n2 * 


For E b /Ng < 2£n2, we always have E bQ (R) < 0, so with BPSK (or QPSK) mod- 
ulation on an AWGN channel, any block coding technique will require an 
E b /No of greater than 10 log^(2£n2) = 1.42 dB for small error rates and 
reasonable implementation complexity, regardless of the code rate or of 
how many quantization levels are used at the demodulator output, as 
stated in [41]. Therefore, we call = ^£n2 the "information bit 

signal energy-to-noise power density ratio threshold", and denote it by 
T. The optimal code rates as a function of E^N^ f° r this example are 
listed in Tables 4.2(a) -4.2(d) and plotted in Figure 4.2. 

The optimal code rates , for BPSK modulation over an AWGN 

channel with demodulator output hard quantization and for MFSK modula- 
tion over an AWGN channel with demodulator output hard quantization are 
listed in Tables 4.3 and 4.4 and depicted in Figures 4.1 and 4.3, re- 
spectively. In the tables we also indicate the E b /Ng threshold, T, for 
each case. 

From these examples we observe that the value of (or is 

relatively small and is inversely proportional to E b /Ng for relatively 

large values of E,/N n . For small values of ( Q r R^^), E../N = 

& b 0 opt ^ opt N 0 

R^^*E,/N„ (or = R^^*E,/N_), and the "channel symbol signal energy-to- 
opt b 0 opt b 0 /& 

noise power density ratio" becomes small compared with E^N^, and conse- 
quently results in a "noisy" channel, or a "higher" channel bit error 
rate. The interesting fact is to note that the "high" bit error rate is 
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Table 4.2(a) . R 


(bO) 

opt 


for BPSK Modulation over 


an AWGN Channel with no Output Quantization 



V N 0 

RC b 0) 

opt 

R o 

E f r C t>0) 
E bO tR opt } 


1.39 (T) 

0 

0 

0 

2.0 

0.317 

0.386 

0.069 

3 . 0 

0.401 

0.621 

0.220 

3.15 

0.402 

0.635 

0.233 

4.0 

0.391 

0.726 

0.335 

5.0 

0.365 

0.784 

0.419 

6.0 

0.339 

0.823 

0.484 

7.0 

0.315 

0.849 

0.534 

8.0 

0.294 

0.869 

0.575 

9.0 

0.276 

0.884 

0.608 

10.0 

0.260 

0.897 

0.637 

11.0 

0.245 

0.906 

0.661 

12.0 

0.233 

0.915 

0.682 

13.0 

0.221 

0.921 

0.700 

14.0 

0.211 

0.927 

0.716 

15.0 

0.202 

0.932 

0.730 
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Table 4.2(b). ^opt^ ^ or Modulation over 

an AWGN Channel with no Output Quantization 



E b /N 0 

R (b0) 

opt 

R o 

E fR (b0) l 
dO opt ] 


1.39 (T) 

0 

0 

0 

2.0 

0.634 

0.772 

0.138 

3.0 

0.802 

1.242 

0.440 

3.15 

0.804 

1.270 

0.466 

4.0 

0.782 

1.452 

0.670 

5.0 

0.730 

1.568 

0.838 

6.0 

0.678 

1.646 

0.968 

7.0 

0.630 

1.698 

1.068 

8.0 

0.588 

1.738 

1.150 

9.0 

0.552 

1.768 

1.216 

10.0 

0.520 

1.794 

1.274 

11.0 

0.490 

1.812 

1.322 

12.0 

0.466 

1.830 

1.364 

13.0 

0.442 

1.842 

1.400 

14.0 

0.422 

1.854 

1.432 

15.0 

0.404 

1.864 

1.460 


4.15 


Table 4.2(c). R^ b ^ for 8-ary PSK Modulation over 
opt 

an AWGN Channel with no Output Quantitation 


V N 0 


,(b0) 

opt 


E (R (b0) l 
£ bO tR opt 3 


1.59 (T) 

0 

0 

0 

2.0 

0.645 

0.783 

0.138 

5.0 

0.855 

1.308 

0.453 

4.0 

0.879 

1.584 

0.705 

5.0 

0.864 

1.764 

0.900 

6.0 

0.846 

1.902 

1.056 

7.0 

0.825 

2.010 

1.185 

8.0 

0.807 

2.100 

1.293 

9.0 

0.792 

2.178 

1.386 

10.0 

0.780 

2.250 

1.470 

11.0 

0.765 

2.310 

1.545 

12.0 

0.753 

2.364 

1.611 

15.0 

0.741 

2.412 

1.671 

14.0 

0.726 

2.451 

1.725 

15.0 

0.714 

2.487 

1.773 
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Table 4.2(d). for 16-ary PSK Modulation over 

opt ' 

an AWGN Channel with no Output Quantization 



V N o 

R (b°) 

opt 

R o 

f rR (b0) i 
E bO CR opt ) 


1.39 (T) 

0 

0 

0 

O 

(N 

0.64 

0.780 

0.140 

3.0 

0.84 

1.292 

0.452 

4.0 

0.88 

1.584 

0.704 

5.0 

0.88 

1.780 

0.900 

6.0 

0.84 

1.896 

1.056 

7.0 

0.84 

2.024 

1.184 

8.0 

0.80 

2.092 

1.292 

9.0 

0.80 

2.188 

1.388 

10.0 

0.80 

2.272 

1.472 

11.0 

0.80 

2.348 

1.548 

12.0 

0.76 

2.376 

1.616 

13.0 

0.76 

2.436 

1.676 

14.0 

0.76 

2.492 

1.732 

15.0 

0.76 

2.548 

1.788 
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Table 4.3. for BPSK Modulation over an 

opt 

AWGN Channel with Output Hard Quantization 



V N 0 

R (b0] 

opt 

R o 

bO^ opt ) 


2.1 CT) 

0 

0 

0 

3.0 

0.26 

0.308 

0.048 

4.0 

0.36 

0.500 

o 

T t 

o 

5.0 

0.38 

0.604 

0.224 

6.0 

0.38 

0.674 

0.294 

7.0 

0.37 

0.722 

0.352 

8.0 

0 . 36 

0.761 

0.401 

9.0 

0.34 

0.782 

0.442 

10.0 

0.33 

0.808 

0.478 

11.0 

0.32 

0.829 

0.509 

12.0 

0.31 

0.846 

0.536 

13.0 

0.29 

0.850 

0.560 

14.0 

0.28 

0.861 

0.581 

15.0 

0.27 

0.870 

0.600 


i 

» 

\ 


1 
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Table 4.4(a). f 0 r BFSK Modulation over an 

opt 

AIVGN Channel with Output Hard Quantization 



V N 0 

R (b0) 

opt 

R o 

E (R (b0) l 
E b<Topt ‘ 


9.5 (T) 0 0 0 


10.0 

0.49 

0.508 

0.018 

12.0 

0.50 

0.609 

0.109 

15.0 

0.48 

0.699 

0.219 

20.0 

0.43 

0.781 

0.351 

25.0 

0.39 

0.832 

0.442 

30.0 

0.35 

0.859 

0.509 

35.0 

0.32 

0.881 

0.561 

40.0 

0.29 

0.892 

0.602 

45.0 

0.27 

0.905 

0 . 635 

50.0 

0.25 

0.913 

0.663 

55.0 

0.24 

0.927 

0.687 

60.0 

0.22 

0.927 

0.707 

65.0 

0.21 

0.934 

0.724 

70.0 

0.20 

0.940 

0.720 

75.0 

0.19 

0.943 

0.753 

80.0 

0.18 

0.945 

0.765 
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Table 4.4(b). for QFSK Modulation over an 

AWGN Channel with Output Hard Quantization 



V N 0 

RC b 0) 

opt 

R o 

E rR Cb0) i 
E b0 opt -* 


5.5 (T) 

0 

0 

0 

6.0 

1.04 

1.088 

0.048 

7.0 

1.06 

1.272 

0.212 

8.0 

1.04 

1.392 

0.352 

8.5 

1.02 

1.434 

0.414 

10.0 

0.96 

1.534 

0.574 

1S.0 

0.78 

1.708 

0.928 

20.0 

0.66 

1.792 

1.132 

25.0 

0.56 

1.828 

1.268 

3Q.0 

0.50 

1.864 

1.364 

35 . 0 

0.44 

1.876 

1.436 

40.0 

0.40 

1.894 

1.494 

45.0 

0.36 

1.898 

1.538 

50.0 

0.34 

1.916 

1.576 

55.0 

0.32 

1.928 

1.608 

60.0 

0.30 

1.934 

1.654 

65.0 

0.28 

1.938 

1.658 

70.0 

0.26 

1.938 

1.678 

80.0 

0.24 

1.950 

1.710 
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Table 4.4(c). for 8-ary FSK Modulation over 

an A1VGN Channel with Output Hard Quantization 



V N o 

D ( b0 ) 

opt 

R o 

e rR Cb0) i 
E bO C opt 1 


4.5 CT) 

0 

0 

0 

5.0 

1.71 

1.881 

0.171 

6.0 

1.65 

2.130 

0.480 

8.0 

1.47 

2.400 

0 .930 

10.0 

1.29 

2.529 

1.239 

13.0 

1.11 

2.664 

1.554 

15.0 

0.99 

2.694 

1.704 

18.0 

0.87 

2.745 

1.875 

20.0 

0.81 

2.775 

1.965 

25.0 

0.69 

2.823 

2.133 

30.0 

0.60 

2.853 

2.253 

35.0 

0.54 

2.880 

2.340 

40.0 

0.48 

2.889 

2.409 

45.0 

0.45 

2.913 

2.463 

50.0 

0.42 

2.928 

2.508 

55.0 

0.39 

2.934 

2.544 

60.0 

0.36 

2.937 

2.577 

65.0 

0.33 

2.937 

2.604 

70.0 

0.30 

2.928 

2.628 

80.0 

0.27 

2.937 

2.667 
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Table 4.4(d). for 16-ary FSK Modulation over 

opt ' 

an AWGN Channel with Output Hard Quantization 



V N 0 

R CbO) 

opt 

R o 

E (R (b0) l 
E bO C opt •* 


3.5 (T) 

0 

0 

0 

4.0 

2.44 

2.464 

0.024 

5.0 

2.36 

2.924 

0.564 

6.0 

2.16 

3.136 

0.976 

8.0 

1.84 

3.392 

1.552 

10.0 

1.60 

3.536 

1.936 

13.0 

1.32 

3.640 

2.320 

15.0 

1.20 

3.704 

2.504 

20.0 

0.96 

3.772 

2.812 

25.0 

0.80 

3.812 

3.012 

30.0 

0.72 

3.872 

3.152 

35.0 

0.64 

3.892 

3.252 

40.0 

0.56 

3.892 

3.332 

45.0 

0.52 

3.916 

3.396 

50.0 

0.48 

3.928 

3.448 

55.0 

0.44 

3.932 

3.492 

60.0 

0.40 

3.928 

3.528 

65.0 

0.36 

3.916 

3 . 556 

75.0 

0.32 

3.928 

3.608 







compensated for by the powerful error correcting capability of the codes 

of rate for R^^), and the net effect is that we obtain the lowest 

opt v opt 

decoding error probability, or equivalently, the largest coding gain. 

The trade off is in a reduction of the system throughput. However, a 
relatively small optimal code rate need not reduce throughput if addi- 
tional bandwidth is available on the channel. Satellite and deep space 
channels, in particular, are not nearly as much bandwidth limited as 
they are power limited. 

In the practically interesting range of values for E^/N^, the 
values of R^j^ (jor remain quite stable. For example, in Figure 

4.1, for BPSK modulation with output hard quantization, when E^/Nq e 
(5,9), which approximately corresponds to a channel bit error rate e = 

10 ^ - 10 the optimal code rate R^^ = 0.321 ± 0.045. This fairly 
stable optimum code rate shows that optimum system performance is not 
overly sensitive to the channel noise. 

Note that E^CR) 0 for 0 _< R <_ C, while E^gCR) JL 0 only for 
0 < R < R q [21,22]. The channel capacity C is the absolute upper limit 
on the rate of a code. The channel cutoff rate R^ is the upper limit 


for practically implementabl e systems [42]. Following the same line of 

bel 
CbO) 


reasoning, we believe that R^^. is the upper limit on the optimal code 


rate, while R^”' more closely matches the real optimal code rate. For 
BPSK modulation on an AWGN channel with demodulator output hard quanti- 

For 


zation. Figure 4.1 shows the difference between R^ and R^ 3 ^ . 

* 6 opt opt 

large E^/N^, they are the same. This fact can be explained as follows. 

It has been shown [21,22] that for small R, E, (R) = E, _(R) . As E./N- 

D bU b U 

becomes large we have seen that both R^ and r^ 0) b ecomes small. 

6 opt opt 

Therefore, for large E^/Nq, the optimal code rates found in terms of 


4.25 


both E^CR) and E, (R) must be the same. 


4.2.2. The Optimal Code Rate in Terms of the Minimum Distance Bound 


For an (n,k) binary block code with minimum distance d . , we de- 

7 nun 

fine the asymptotic coding gain in the hard quantization case [see 
Appendix C] as 


R*d . 

y = 10 log 1Q dB , 


(4.25) 


for large E^/N^. The asymptotic lower and upper bounds on d^^ are 


[43] 


d • i 

i * mm TI -i 

lim — — > H (1-R) , 


n-*» 


n 


(4.24) 


where HCx) = -x log 2 x - (1-x) log 2 (l-x) , for 0 <_ x _< y, is the binary 
entropy function, and 
d 


. . min _ 1 .... 
lim — — < -r(l-R) , 


n-*=° 


n - 2 


(4.25) 


respectively. 

Define 


F b tR) 


= R*lim 

n-H» 


d . 
mm 


(4.26) 


From (4.23) we see that the optimal code rate in terms of the minimum 

find 1 

distance bound, denoted by ' ‘ , should maximize (4.26). By numerical 

calculation, we find that 


R 


(bd) _ _ 
opt 


0.3778, from the lower bound (4.24), 
0.5 , from the upper bound (4.25) 


(4.27) 


These results show that for large E^/Nqj the optimal code rate 



Ro pt e [°- 5778 > °- 5 ]- 

An apparently contradictory fact is that the optimal code rate 


4.24. 


found in terms of the error probability bounds is a function of E^/Nq, 
while the optimal code rate found by using minimum distance bounds is a 
fixed number for large E^/N^. If one notice that the coding gain of 
(4.23) are defined by using only the first term of the decoding error 
probability expression [see Appendix C] , the data become comprehensive. 


4.3. The Optimal Convolutional Code Rate 

4.3.1. The Optimal Code Rate in Terms of the Error Probability Bound 


The channel coding theorem for convolutional codes states that 
[21]: For a Q-ary input DMC, there exists an (n,k,m) time-varying con- 

volutional code of constraint length n^ = nCl+m) , and arbitrary sequence 
length, whose bit error probability P^, resulting from maximum- likeli- 
hood decoding, is bounded by 


p b 1 C2 k -1) — 


-n A E c C R ) 


6 > 0 , 


-6nE (R) 
[ 1-2 C ] 


E c CR) = R q , 0<R<R 0 O-6), 

( E fR) = max EC P, £) , 0 < p < 1, 

a 

EqCp,£) 

^ R = (l-6)max — , R Q C1-S) 1 R 1 C(l-6), 


(4.28.1) 


(4.28.2) 


(4.28.3) 


where <S is an arbitrary positive number, and Eq(p, £) and R are given 


fc’! 

in C4.1) and C4.8), respectively. We define the optimal code rate R^ ' 


4.:s 


as the code rate R which 


maximises: E (R), 

c 


subject to: E c (R) > 0. 


(4.29) 


If the value of R is restricted only to 0 <_ R <_ Rq( 1-6), from 
(4.28.1) and (4.28.2) we obtain 


P b < C 2 k -D -1 


-\ E coW 


, 6 > 0 , 


-<5nE (R) ' 

[ 1-2 c0 ] 


(4.30.1) 


E c 0 CR) = R 0 , o £ R 1 R Q (l-<5) 


(4.30.2) 


In this case the optimization problem is 


maximize: E «(R) , 

cO 


subject to: E c g(R) > 

and the code rate R which satisfies (4.31) is denoted by R 


(4.31) 


CcO) 
opt ’ 


For most real channels, Eq(p, 3.) (and consequently Rq) is an in- 
creasing function of E^/Nq s R'E^/Nq. Therefore, without loss of 
generality, we can make the following assumption: 


Eg(p, £) (and Rq) is an increasing function of R for fixed 
E^/Nq and an increasing function of E^/Nq for fixed R. 

Under this assumption, the optimization problems of (4.29) and (4.31) 
can be reduced to: 


R^ = max R, 
0pt 0<R<C 


subject to: E c CR) > 0, 


(4.32) 


and 


4.26 • 


R opt } = max R ’ 
F °1 R 1 R 0 


subject to: E c g(R) > 0> 


(4.33) 


respectively. 

(c) (cO) 

Note that ' is always greater than or equal to . Also, by 

our assumption, ( R) and E c ^(R) are increasing functions of E^/N^ for 
fixed R, and this implies that the optimal code rate is proportional to 
E^/Nq. Because E^fR) and E c q(R) approach Ic^Q on a Q~ary input DMC as 
E^/Np goes to infinity, the optimal code rate R^j^ (or R opt^ also 
approaches log^Q asymptotically. The optimal code rates as a function 
of E b /N Q for the same modulation schemes considered for block codes are 
given in Tables 4. 5-4. 8 and shown graphically in Figures 4. 4-4. 6. 

Note that in Figures 4. 4-4. 6, the optimal convolutional code rate 
is very close to lc^Q in the practically interesting range of E^/N^. 
This implies that no coding is necessary. The reason for obtaining such 
a rather surprising result is that the convolutional coding exponents 
E c (R) and E c ^(R) are not as accurate as the block coding exponents E^(R) 
and E^q(R) at low code rates. To obtain a more meaningful optimal code 
rate, a tighter bound on decoding error probability at low code rates - 
the expurgated upper bound - should be used. The expurgated bound for 
convolutional codes states that [21]: For binary-input, output -symmetric 
channels, there exists a time-varying convolutional code of constraint 
length n and rate R = k/n bits per channel use for which the bit error 
probability with maximum likelihood decoding satisfies 
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Table 4.5. for BPSK Modulation over an 

opt 

AWGN Channel with Output Hard Quantitation 



V N o 

R (c) 

opt 

C 

E (R Cc) l 
c opt' 


1.0 (T) 

0 

0 

0 

1.1 

0.01 

0.0101 

0.0001 

1.2 

0.09 

0.0945 

0.0048 

1.3 

0.15 

0.1641 

0.0159 

1.5 

0.26 

0.3015 

0.0525 

2.0 

0.42 

0.5391 

0.1924 

2.5 

0.51 

0.6921 

0.3652 

3.0 

0.65 

0.7888 

0.5511 

3 . 5 

0.70 

0.8973 

0.7011 

4.0 

0.79 

0.9473 

0.7933 

4.5 

0.85 

0.9719 

0.8541 

5.0 

0.89 

0.9845 

0.8950 

5.5 

0.92 

0.9913 

0.9239 

6.0 

0.94 

0.9950 

0.9440 

6.5 

0.95 

0.9970 

0.9578 

7.0 

0.96 

0.9982 

0.9683 

7.5 

0.97 

0.9990 

0.9764 

8.0 

0.98 

0.9994 

0.9824 

9.5 

0.99 

0.9999 

0.9923 
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Table 4.6(a) . R 


(cO) 

opt 


for BPSK Modulation over 


an A.WGN Channel with no Output Quantization 



V N 0 

R CcO) 

opt 

E cO tR £? K V 


1.3 CT) 

0 

0 

1.4 

0.02 

0.0201 

1.5 

0.20 

0.2002 

1.7 

0.44 

0.4409 

2.0 

0.65 

0.6523 

2.3 

0.77 

0.7733 

2.5 

0.82 

0.8253 

3.0 

0.90 

0.9062 

3.5 

0.94 

0.9472 

4.0 

0.97 

0.9705 

4.5 

0.98 

0.9826 

5.5 

0.99 

0.9938 
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Table 4.6(b). 


R^pt^ for QPSK Modulation over 


an AWGN Channel with no Output Quantization 



V N 0 

nCcO) 

opt 



1.3 (T) 

0 

0 

1.4 

0.04 

0.0402 

1.5 

0.40 

0.4004 

1.7 

0.88 

0.8818 

2.0 

1.30 

1.3046 

2.3 

1.54 

1.5466 

2.5 

1.64 

1.6506 

3.0 

1.80 

1.8124 

3.5 

1.88 

1.8944 

4.0 

1.94 

1.9410 

4.5 

1.96 

1.9652 

5.5 

1.98 

1.9876 
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Table 4 .6(c) . R 


(cO) 

opt 


for 8-arv PSK Modulation over 


an AWGN Channel with no Output Quantisation 



V N o 

R (C0) 

opt 

E n CR fc ^)CR n ) 

cCr opt J v 0 J 


1.5 (T) 

0 

0 

1.4 

0.03 

0.0303 

1.5 

0.39 

0.3912 

1.8 

1.05 

1.0575 

2.0 

1.35 

1.3527 

2.5 

1.77 

1.7838 

3.0 

2.04 

2.0562 

3.5 

2.25 

2.2569 

4.0 

2.40 

2.4072 

4.5 

2.52 

2.5275 

5.0 

2.61 

2.6223 

5.5 

2.70 

2.7027 

6.0 

2.76 

2.7639 

6.5 

2.79 

2.8095 

7.0 

2.85 

2.8521 

8.0 

2.88 

2.904 

9.0 

2.94 

2.9412 
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Table 4.6(d). for 16-ary PSK Modulation over 

an AWGN Channel with no Output Quatnization 



V N o 

R (c°) 

opt 

E n CR Cc /*) (RO) 
cO opt 1 


1.3 (T) 

0 

0 

1.4 

0.04 

0.0404 

1.5 

0.40 

0.4004 

1.7 

0.88 

0.8840 

2.0 

1.32 

1.3332 

2.5 

1.76 

1.7788 

3.0 

2.04 

2.0564 

3.5 

2.24 

2.2556 

4.0 

2.40 

2.4136 

4.5 

2.52 

2.5412 

5.0 

2.64 

2.656 

5.5 

2.76 

2.7612 

6.0 

2.84 

2.8472 

6.5 

2.92 

2.9276 

7.0 

3.00 

3.0024 

7.5 

3.04 

3.0632 

8.0 

3.12 

3.1300 

9.0 

3.24 

3.2436 
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Table 4.7. for BPSK Modulation over an 

opt 

AWGN Channel with Output Hard Quantization 



V N o 

R CC0) 

opt 



2.1 (T) 

0 

0 

2.2 

0.02 

0.0201 

2.5 

0.28 

0.2810 

3.0 

0.55 

0.5503 

3.5 

0.70 

0.7011 

4.0 

0.79 

0.7933 

4.5 

0.85 

0.8541 

5.0 

0.89 

0.8950 

6.0 

0.94 

0.9440 

6.5 

0.95 

0.9578 

7.0 

0.96 

0.9683 

7.5 

0.97 

0.9764 

8.0 

0.98 

0.9824 

9.5 

0.99 

0.9923 
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Table 4.3(a). R 


(cO) 

opt 


for BFSK Modulation over an 


AWGN Channel with Output Hard Quantization 



V N o 

R (C0) 

opt 

E cO (l C ,KI V 


9.5 (T) 0 0 


10.0 

0.64 

0.6409 

10.5 

0.72 

0.7221 

11.0 

0.77 

0.7740 

11.5 

0.81 

0.8142 

12.0 

0.84 

0.8448 

13.0 

0.89 

0.8912 

14.0 

0.92 

0.9207 

15.0 

0.94 

0.9411 
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Table 4.8(b) . 
AWGN Channel 


R opt^ ^ or Modulation over an 

with Output Hard Quantization 



V N o 

R CC0) 

opt 

E c0 CI O<V 


5.5 (T) 

0 

0 

6.0 

1.40 

1.4012 

6.5 

1.60 

1.6086 

7.0 

1.72 

1.7288 

7.5 

1.80 

1.8064 

8.0 

1.86 

1.8602 

8.5 

1.88 

1.8926 

9.0 

1.92 

1.9214 

10.0 

1.94 

1.9532 

11.0 

1.96 

1.9726 

12.0 

1.98 

1.9840 
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Table 4.8(c). R 


(cO) 

opt 


for 8-arv FSK Modulation over 


an AWGN Channel with Output Hard Quantization 



V N 0 

R CcO) 

opt 

E cO (R op? >K V 


4.5 (T) 

0 

0 

5.0 

2.46 

2.4639 

5.5 

2.67 

2.6817 

6.0 

2.79 

2.8011 

6.5 

2.85 

2.8695 

7.0 

2.91 

2.9163 

7.5 

2.94 

2.9442 

8.5 

2.97 

2.9745 


4.38- 


,CcO) 


Table 4.8(d). 

an AWGN Channel with Output Hard Quantization 


for i6_ ar y pgK Modulation over 
opt } 



V N o 

R (c0) 

opt 

E cO (R op? ))( V 


3.5 (T) 

0 

0 

4.0 

2.76 

2.7648 

4.5 

3.52 

3.5208 

5.0 

3.72 

3.7404 

5.5 

3.82 

3.8556 

6.0 

3.88 

3.9140 

6.5 

3.92 

3.9496 

7.0 

3.96 

3.9708 


1 



Optimal 
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P b < 


2 k -l 


' E cex (R)/[R(1 + <5)] 


( 1-2 


-b S)‘ 


-n.E (R) 
A cex v J 


(4.34.1) 


where 


E 

cex 


(R) 


R(l+<5) JinZ 

.n[2- R ^ 1+6 )-l] ’ 


0 1 R 1 R Q /(l+<5), 


(4.34.2) 


6 > 0 is an arbitrary positive number, and 


Z = l v y P(y|0)p(y|l) . 

y 


(4.34.3) 


Since the exponential term dominates the error probability express 
the optimal code rate, denoted by 




> ( cex ), satisfies 


maximizing: E (R) , 

C 0X 


subject to: E (R) > 0. 

C6X 


(4.35) 


The optimal code rates as a function of E^/N^, for BPSK and BFSK modu- 
lation on an AWGN channel with output hard quantization are shown in 
Tables 4.9 and 4.10 and plotted in Figures 4.4 and 4.6, respectively. 
Note that R^® X ^ is around 0.75 for large E^/N^. 


4.3.2. The Optimal Code Rate in Terms of the Free Distance Bound 

With maximum- likelihood decoding of convolutional codes, for large 
E^/Ng, the asymptotic coding gain is given by [2] 

R*d 

y = 10 l°gjQ — -~ e - dB , for hard quantization, (4.36) 
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Table 4.9. R^ cex ^ f or BPSK Modulation over an 
opt 

AWGN Channel with Output Hard Quantisation 



V N 0 

R (cex) 

opt 

R o 

E CR (c ® x) ) 

cex opt 


2.2 (T) 

0.02 

0.0201 

0.0201 

2.3 

0.12 

0.1205 

0.1203 

2.4 

0.21 

0.2102 

0.2102 

2.5 

0.28 

0.2810 

0.2811 

2.6 

0.35 

0.3506 

0.3507 

2.7 

0.41 

0.4104 

0.4105 

2.8 

0.46 

0.4610 

0.4612 

2.9 

0.50 

0.5028 

0.5038 

3.0 

0.55 

0.5503 

0.5505 

3.2 

0.62 

0.6204 

0.6206 

3.4 

0.67 

0.6734 

0.6758 

3.6 

0.72 

0.7222 

0.7241 

4.0 

0.75 

0.7753 

0.8021 

5.0 

0.75 

0.8482 

0.9906 

10.0 

0.75 

0.9790 

1.9019 

15.0 

0.75 

0.9970 

2.7881 
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Table 4,10. for BFSK Modulation over an 

opt 

AWGN Channel with Output Hard Quantitation 



V N o 

R (cex) 

opt 

R o 

E (R (c ® x) ) 

cex^ opt 


9.65 (T) 

0.51 

0.5102 

0.5102 

9.70 

0.54 

0.5408 

0.5412 

9.90 

0.62 

0.6201 

0.6201 

10.0 

0.64 

0.6410 

0.6415 

10.3 

0.69 

0.6925 

0.6944 

10.5 

0.72 

0.7221 

0.7239 

10.8 

0.76 

0.7601 

0.7605 

11.0 

0.77 

0.7740 

0.7784 

11.5 

0.79 

0.8040 

0.8215 

15.0 

0.78 

0.8945 

1.1155 

20.0 

0.78 

0.9593 

1.5369 

25.0 

0.77 

0.9835 

1.9590 

30.0 

0.77 

0.9937 

2.3811 

35.0 

0.77 

0.9976 

2.8033 

40.0 

0.77 

0.9991 

3.2255 
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Y = 10 log^R'dj. dB , for infinite fine 

& 10 free . „ . ,, 

quantization (4.o/) 

where d^^ is the free distance of the convolutional code. The 
asymptotic bounds on free distance are given by [44] : 


lim 

n .-**» 
A 


lim 

n.-*» 

A 

for systematic 


lim 

n.-x° 

A 


^free ^ 1-R 
n A 2 ’ 

^ > H~ 1 (1-R) , 
n A ~ 

fixed codes , where n^ 

^free 1 
n A 2 ’ 


(4.58.1] 

(4.38.2) 

is the code constraint length; 

(4.39.1) 


lim 

n,-*» 

A 


free 

> 

n. - 


2R(1 - 2 2R ' 1 ) o < R < 3 
H(2 2R -^ ♦ 2R-1 ' " - 8 


2H _1 (1-R) 




(4.39.2) 


for nonsystematic fixed codes; and 


A R_ 1 

, . free ^ R(1 - 2* A ) 

lim > — — — 


n.-H» n A H(2 R_1 ) + R-l 


for nonsystematic time-varying codes. 
Define 

F (R) = R-lim . 

c n.-~» n A 

A 


(4.40) 


(4.41) 


The code rate R which maximizes F C (R) is called the optimal code rate, 

and is denoted by R^ C ^. If follows from (4.36) and (4.37) that R^ c f^ 

opt opt 
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is the code rate which maximizes the coding gain 
The optimization probelm can best be solved 
numerical calculation, we find that 


for large E^/Nq. 
by computer. By 


1) For systematic fixed codes 


rW - 1 

opt 


0.5 , from the upper bound 

0.5778, from the lower bound 


(4.38.1) , 

(4.38.2) ; 


2) For nonsystematic fixed codes 



1 , from the upper bound 

0.3778, from the lower bound 


(4.39.1) , 

(4.39.2) ; 


3) For nonsystematic time varying codes 



= 0.7580, from the lower bound (4.40). 


(4.42) 


(4.43) 


(4.44) 


These results indicate that for systematic fixed codes low code 

rate should be used while for nonsystematic codes high code rate should 

be used for optimum performance. It is interesting to compare (4.44) 

fccx*) ' 

with the optimal code rate Rjj found in terms of the expurgated 
bound, they both indicate that the optimal code rate is around 0.75 for 
large ^/Nq. 


4.4. The Optimal Concatenated Code Rate 

The complexity of conventional coding systems grows exponentially 
with the block length for block codes or with the constraint length for 
convolutional codes. To overcome the complexity of very long codes. 


4.45 


Forney [24] first introduced concatenated codes as a practical means of 
implementing codes with long block or constraint lengths. Concatenated 
coding systems are usually implemented by employing two levels of cod- 
ing, as illustrated in Figure 4.7. Binary data from the data source is 
serially partitioned into k..-bit blocks that are subsequently used as 

input signals to a 2 -ary block encoder known as the outer encoder. 
k l 

Usually Q = 2 -ary RS codes are used for this purpose. The output of 
the RS encoder is converted back into bits and serially encoded by the 
inner encoder, which may be either block or convolutional (only block 
inner codes are considered here), and the resultant sequence of channel 
symbols is sent over the physical channel - the inner channel. Decoding 
is accomplished in the reverse order. 

To evaluate the optimal code rate of the concatenated code 
described above, let us investigate the performance obtainable with the 
following concatenation scheme. The inner code is a block code of 

length n^ and rate R , where R^^ = log 2 M/n 1 , and M is the number of code 

words in the inner code. The inner decoder is a maximum- likelihood 
decoder which puts out an estimate of which code word was sent, with 
average probability of error p. The outer code is a block code of 

length n 2 and rate R 2< Given an estimate, the outer decoder makes no 

distinction between the outputs other than the estimate. Therefore, we 
regard the outer channel as an equierror channel, and the outer channel 
has transition probability matrix 


p(y|x) = 1 


1-p 


n R 

P(2 -1) 


y - x, 
y f x. 


(4.45) 


The overall rate and length of the concatenated code are R = R^R 2 


4 . 


Data 

Source 


Data 

Sink 



L. 


Figure 4.7. 


Concatenated Coding System 
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and N = n^n^, respectively. Define R* = n R. Note that the total num- 
ber of code words in the concatenated code is 2 “ = 2 “ = 2' . 

First, we derive a random coding bound for the concatenated code in 
terms of the channel cutoff rate. Suppose that the best inner code of 
length n^ and rate R^ is used. Then 


P 1 2 


-WV 


°± R 1± R 01> 


(4.46) 


where R^^ is the inner channel cutoff rate. The outer channel cutoff 
rate, denoted by R^* is given by (4.8). By symmetry, the probability 
vector q for which the left hand side of (4.8) is maximized is 


q(x) = 2 


-"i R i 


for all x. 


(4.47) 


Substituting (4.45-4.47) into (4.8), we obtain 


R > lor ' r / , 

Rq 2 I “ lo §2 2 [*'1-2 


n R -n (R ni -R.) 

W (2 1 l -l) 2 1 01 1 ] * 


(4.48) 


As n^ becomes large, (4.48) is reduced to 


R 02 — 


n i R l 


R oi - 2R r 


n i*- R oi“ R P 1 R oi - 2 R i’ 


or in another form 


r 02 - n i min [C R 0 r R i)» Rjl • 


(4.49) 


Now define 




4.4V 


E „o' R > 



>_ n 1 min[ (R q 1 -R 1 ), Rj] - n J R 

= rijtmintCRpj-Rj), Rj ] - R}. 


(4.50) 


In order to maximize performance, for a given overall rate R, and 
may be varied subject to the constraint R = R^R 2 « We then define 


E n (R) = - 
nO v J n. 


max 


R 1 R 2=R 



CR ) 


_> max (min[ (R^-R^) , R^] - 


R), 


r 1 r 2 =r 


(4.51) 


Therefore, we can claim from the coding theorem (in terms of the cutoff 

n R 

rate) that if there exists an equierror outer channel with 2 inputs 
and average error probability p, then there exists a concatenated code 
of overall length N and overall rate R such that 


P E — 2 


-NE n0 C R ) 


, 0 1 R < R or 


(4.52) 


Note that in (4.51), R^ is a function of R^ and R^, and so is E n g(R) 
To emphasize this fact, we rewrite E n ^(R) as E^^R^, R 2 ), and conse- 
quently 


P E - 2 


-NE n0 ( R i, r 2 ) 


, 0 < R < R fll . 


(4.53) 


Now the optimal inner and outer code rates, 
, respectively, should satisfy 

maximize: E -(R^> R 2 ), 

subject to: E n g( R i> > 


denoted by R^^ 
opt 


and 


(4.54) 
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A tighter bound on is given by a concatenated coding theorem in 
Reference 24. But in [241 the inner code rate is restricted to R ,< 

Rj < Cj, where and are the inner channel critical rate and the 
inner channel capacity, respectively. For our optimal code rate 
problem, the inner code rate can be any value between 0 and . This is 
done by slightly modifying Forney’s theorem in [24]. From the block 
coding theorem, the outer channel average probability of error is 
bounded by 


-n 

p < 2 


iW 


0 < R < C^, 


(4.55) 


where E^(R^) is given by (4.5). Using (4.55) and following the same 

line of reasoning as Forney, we obtain: If there exists an equierror 

n R 

outer channel with 2 inputs and average error probability p, bounded 
by (4.55), then there exists a concatenated code of overall length N and 
overall rate R such that 


P E^ 2 


■NE n (R) 


< R < C l, 


(4.56) 


where 


E n (R) = 


max (min[E^(R^) 


r 1 r 2 =r 


RjHl-RpK 


(4.57) 


Again note that E n (R) is, in fact, a function of R^ and R 2 , so we can 
rewrite it as E^tR^, R,,), and (4.56) becomes 




■ n w 


R 2> 


< R ^ . 


(4.58) 


Therefore the optimization problem is 


4.50 


maximize: E (R. , R-), 

n 1 i 

subject to: E^fR^, R 2^ > (4*59) 

and the inner and outer code rates R^ and R^ which satisfy (4.59) are 
denoted by R^^ and respectively. 


Example 4.3 

Assume that the inner channel is a BSC derived from forcing hard 

decisions on an AWGN channel with BPSK modulation. This channel is 

representative of the deep space channel where concatenated codes have 

met with a great deal of success. The numerical values of optimal 

inner and outer code rates in terms of both E ,,(R. , R-) and E (R, . R„) 

nO 1 2' n 1 2 

are listed in Table 4.11 and shown in Figure 4.8 as a function of E^/N 
From Figure 4.8 we see that low inner code rates and high outer code 
rates should be used in such a concatenated coding system. 
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Table 4 
over 

• 11(a). 
an AWGN 

R< 10) a 
opt 

Channel 

nd for BPSK Modulation 

opt 

with Output Hard Quantization 


V N o 

R (10) 

opt 

r (20) 

opt 

R 01 

CIO) (20) 

n0 L opt ’ opt J 

4.4 

0.01 

1.0 

0.0200 

2xio~ 6 

4.5 

0.02 

0.98 

0.0398 

0.0002 

4.7 

0.04 

0.96 

0.0809 

0.0016 

4.9 

0.06 

0.94 

0.1204 

0.0036 

5.1 

0.08 

0.92 

0.1604 

0.0064 

5.3 

0.10 

0.90 

0.1999 

0.0099 

5.5 

0.12 

0.89 

0.2410 

0.0132 

5.7 

0.14 

0.87 

0.2790 

0.0172 

6.0 

0.16 

0.85 

0.3202 

0.0240 

6.4 

0.18 

0.82 

0.3615 

0.0324 

6.7 

0.20 

0.80 

0.4002 

0.0400 

7.1 

0.22 

0.78 

0.4420 

0.0484 

7.6 

0.25 

0.76 

0.5013 

0.0600 

7.9 

0.26 

0.74 

0.5200 

0.0675 

8.5 

0.27 

0.70 

0.5404 

0.0810 

9.1 

0 . 30 

0.69 

0.6005 

0.0930 

9.7 

0.31 

0.66 

0.6205 

0.1054 

10.2 

0.32 

0.64 

0.6402 

0.1152 

11.1 

0.33 

0.60 

0.6597 

0.1317 

11.8 

0.34 

0.58 

0.6815 

0.1428 

12.8 

0.36 

0.56 

0.7206 

0.1584 
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Table 4.11(b). R 
an AWGN Channel 


^ and r!^ for BPSK Modulation over 
opt opt 

with Output Hard Quantization 



V N o 

R (1 J 

opt 

rC 2 ) 

opt 

c i 

E CR C1 > R (2) J 
n opt opt 


1.2 

O'. 05 

0.98 

0.0522 

1 * 10 -6 

1.3 

0.08 

0.96 

0.0870 

6 xl 0 -6 

1.4 

0.11 

0.94 

0.1237 

2 . 6 xl 0 -5 

1.5 

0.14 

0.93 

0.1634 

7 . 1 xio ' 5 

-4 

1.6 

0.16 

0.91 

0.1917 

1 . 5 x 10 

1.7 

0.18 

0.90 

0.2226 

2 . 7 x IQ -4 

1.8 

0.20 

0.89 

0.2540 

0.0004 

2.0 

0.22 

0.86 

0.2928 

0.0009 

2.1 

0.24 

0.86 

0.3277 

0.0012 

2.3 

0.26 

0.84 

0.3690 

0.0021 

2.5 

0.28 

0.82 

0.4093 

0.0031 

2.8 

0.30 

0.80 

0.4604 

0.0051 

3.2 

0.32 

0.78 

0.5201 

0.0084 

5.8 

0.34 

0.75 

0.5900 

0.0147 

4.3 

0.35 

0.73 

0 . 6365 

0.0211 

5.0 

0.36 

0.71 

0.6923 

0.0312 

5.75 

0.37 

0.69 

0.742 7 

0.0435 

6.75 

0.38 

0.67 

0.7963 

0.0613 

7.75 

0.38 

0.64 

0.8256 

0.0795 

8.75 

0.38 

0.61 

0.8475 

0.0971 

9.75 

0.38 

0.58 

0.8637 

0.1138 
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